• View
• Edit
• History
• Print

Lab1

General Remarks

• We suggest to read the following tutorials before you start implementing:
  • Networking Basics: Short explanation of networking basics like TCP, UDP and ports.
  • Java IO Tutorial: It's absolutely necessary to be familiar with I/O and streams to do sockets operations! You can skip it, if you are already familiar with it.
  • Java TCP Sockets Tutorial: A very useful introduction into (TCP) sockets programming.
  • Java Datagrams Tutorial: Provides all the information you need to send and receive datagram packets using UDP.
  • Java Concurrency Tutorial: A tutorial about concurrency that covers threads, thread-pools and synchronization.
• Group work is NOT allowed in the lab. You have to work alone. Discussions with colleagues (e.g., in the forum) are allowed but the code has to be written alone.
• Be sure to check the Tricky Parts section for questions!

Submission Guide (for all Labs)

Submission

• You must upload your solution using the Teaching Tool before the submission deadline: 6.11, 18:00 cet. - the deadline is hard! You are responsible for submitting your solution in time. If you do not submit, you won't get any points!
• Upload your solution as a ZIP file. Please submit only the sources of your solution and the build.xml file (not the compiled class files and no third-party libraries).
• Your submission must compile and run in our lab environment. Use and complete the provided ant template.
• Before the submission deadline, you can upload your solution as often as you like. Note that any existing submission will be replaced by uploading a new one.
• Please make sure that your upload was successful (i.e., you should be able to download your solution - as the tutors will do during the interview).

Interviews

• After the submission deadline, there will be a mandatory interview (Abgabegespräch). You must register for a time slot to the interviews using the Teaching Tool.
• You can do the interview only if you submitted your solution before the deadline!
• The interview will take place in the DSLab. During the interview, you will be asked about the solution that you uploaded (i.e., changes after the deadline will not be taken into account!). In the interview you need to explain your code, design and architecture in detail.
• Remember that you can do the interview only once!

Description

In this assignment you will learn

• the basics of TCP and UDP socket communication
• how to program multithreaded
• different connection types

Overview

This year we are going to build a centralized peer-to-peer file sharing system. The architecture is the following:
There is one central server that is responsible for managing peer and file information by mapping shared files to owning peers as well as mapping peers to their shared files. Each peer is identified by its IP address and a TCP port, at which the peer listens for incoming file download requests. Each file is identified by its name and size (this is of course a simplification, but it should be good enough for our file sharing). Peers have to register at the server so they can share files and search for files by its name or peer. The real file transmission is done on a peer-to-peer basis. Figure 1 illustrates a simple example:

Figure 1

There is no connection being held between the server and the client between two distinct requests, since peers may be online for a very long time while downloading or uploading, but without submitting requests to the server. This would lead to a waste of resources, so, instead, peers must send UDP packets from time to time to the server, to signal that the peer is still online (so called "isAlive" packets). It also implies that peers have to submit their TCP port on each server communication. This is shown in Figure 2: While one peer regularly sends isAlive packets to the server, the other peer fails to send these packets. Therefore, this peer will be removed by the server.

Figure 2

Please note that these figures are for illustration purposes and some details have been omitted for the sake of simplicity. You find these details (parameters, return values etc.) below.

Server

Arguments

Your server program shall expect the following arguments:

• tcpPort: the port to be used for instantiating a java.net.ServerSocket (handling TCP requests).
• udpPort: the port to be used for instantiating a java.net.DatagramSocket (handling UDP requests).
• peerTimeout: the period in ms each peer has to send an “isAlive” packet (only containing the peer's TCP port). If no such is received for that time the peer is assumed to be offline and is removed.
• checkPeriod: is tightly connected to the peerTimeout argument. It specifies the time in ms between two peer checks (checking whether the last “isAlive” packet was received more than peerTimeout ms ago).

If any argument is invalid or missing, print a usage message and exit.

Implementation details

Create a java.net.ServerSocket (=ServerSocket) as well as a java.net.DatagramSocket (=DatagramSocket) instance.
We want to concurrently listen for new connections on the ServerSocket and wait for incoming packets on the DatagramSocket. The ServerSocket listens for incoming messages (described in Interactive commands) whereas the DatagramSocket listens for incoming isAlive packets. Since both relevant methods (ServerSocket.accept() und DatagramSocket.receive()) are blocking operations, you shall spawn a new thread for each, in which you call these methods in a loop.

Since java.net.Sockets (=Socket), which are returned by ServerSocket.accept(), provide blocking I/O-operations (via getInputStream() and getOutputStream()) and we want to serve multiple peers simultaneously, again each incoming request shall be handled in an own thread. To maximize the concurrency and performance of our server also each incoming java.net.DatagramPacket (filled by DatagramSocket.receive()) shall be processed in an own thread. Study the java sockets and datagrams tutorial to get familiar with these constructs.
We recommend using thread pools (implementations of java.util.concurrent.ExecutorService) for implementing this behavior. They help to minimize the overhead of creating a thread every time a request is received by reusing already existing thread instances. Java provides some sophisticated implementations that can be easily instantiated by using the static factory methods of java.util.concurrent.Executors. Anyway you may also manually instantiate new threads on your own without using these classes. Help can be found in the java concurrency tutorial.

A word about peer and file management. When a peer registers, it submits information about all its shared files. The server then has to find out whether any of the submitted files are already shared by other peers. As already stated, two files should be equal if their names and sizes are equal. If one already exists the new peer must of course be added to the peers owning that file. When a peer unregisters (or is removed by the peer garbage collector, see below) all its shared files must be checked. If the peer is the only or last online owner of a file, that file should of course not be found by any other peer any more. So develop an efficient data structure for accessing/removing peers and files.
Peers who did not send an “isAlive” packet for the specified time (peerTimeout plus 500ms extra leeway), must be removed. You can either use a thread or a java.util.Timer in combination with a java.util.TimerTask to implement the peer garbage collector, which checks the peers every checkPeriod ms. On removal of a peer you shall print "Peer <ipAddress:port> timed out." to the console.
For any incoming request (except registration requests, of course) the server has to check whether the requesting peer is registered. If not, you may simply close the connection.
Since you've got to manage peers and files across threads you have to deal with synchronization – make sure your code is thread-safe. Study the java concurrency tutorial if you are not familiar with threading and/or synchronization.

As soon as your server is ready for serving requests, you shall print “Server up. Hit enter to exit.” to the console and of course implement this behavior. Note that as long as there is any non-daemon thread alive, the application won't shut down, so you need to stop them. Therefore call ServerSocket.close(), which will throw a java.net.SocketException in the thread blocked in ServerSocket.accept(), and DatagramSocket.close(), which will throw a java.net.SocketException in the thread blocked in DatagramSocket.receive(). All other threads currently alive should simply run out. If you are using a java.util.concurrent.ExecutorService you have to call its shutdown() method. Anyway you may not call System.exit(), instead free all acquired resources orderly.

Peer

In this part you have to build an interactive command line peer application.

Arguments

Your peer program shall take four command line arguments:

• tcpPort: the port to be used for instantiating a ServerSocket (handling TCP file uploads).
• udpPort: the ports to be used for instantiating a DatagramSocket (sending "isAlive" packets).
• serverHost:serverPort: a pair consisting of a host name (or IP address) and a port, defining where the server is listening for (TCP) requests.
• sharedDir: the directory where shared files are located.

If any argument is invalid or missing print a usage message and exit.

Implementation details

On startup the peer shall create a ServerSocket and a DatagramSocket. You also have to scan the shared directory for existing files (subdirectories don't have to be considered). If the directory does not yet exist, create it. Note that it's not required to observe the shared directory: newly (externally) inserted files don't have to be recognized as shared files; also deleted files don't have to be detected. Thus you also don't have to inform the server about such updates.

The peer then has to register at the server by establishing a connection to it by creating a new Socket instance. The peer submits its own TCP port and all information (i.e. the names and sizes) about its shared files, while the server returns its UDP port and the period in ms for sending "isAlive" packets. If the registration was successful, print "Registration successful." to the console, otherwise exit the application.

From now on the peer has to send "isAlive" packets (only containing the peer's TCP port, so the server can associate the packet with the peer) to the server every period ms. Again you can use a java.util.Timer with a java.util.TimerTask or another thread to send these packets (i.e. java.net.DatagramPackets via DatagramSocket.send()) periodically. Note that no user interaction must be required!

Now you know why you have to create a DatagramSocket, but what's the ServerSocket's purpose? Since we are creating a P2P file transfer application, the shared files must be transferred directly between two peers. Therefore one peer (who owns the requested file) has to become some sort of “server”, the other one (who’s requesting the file) a client. The server peer has to provide a ServerSocket to which the client peer can connect (i.e. create a Socket). Then the file transmission may start: the server peer is uploading (writing data into the socket), while the client peer is downloading (reading data from the socket). Since every peer might become a "server peer" it has to create a ServerSocket at startup and start listening for file requests, once again, in a new thread. Again each incoming request has to be handled in an own thread, so that multiple requests can be served at the same time.

Please note that in any server communication the peer has to submit its TCP port (where the ServerSocket is listening), thus allowing the server to identify the peer and check it’s registered. If the server is not online (i.e. creating the Socket fails), provide some meaningful output and exit the application.

Interactive commands

The following commands must be supported by your peer application. Take care of handling invalid commands and arguments and provide usage messages in these cases.

• findFile <fileName> [-e]

• filesOfPeer <ipAddress:port>

• downloadFile <ipAddress:port> <fileName>

• stop

Lab port policy

Since it's not possible to open a ServerSocket or DatagramSocket on ports where already another TCP or UDP service is listening for requests, we have to make sure each students uses its own port range.
So if you are testing your solution in the lab environment (i.e. on the lab servers) you have to obey the following rule: you may only use ports between 10.000 + dslabXXX * 10 and 10.000 + (dslabXXX + 1) * 10 - 1. So if your account is dslab250 you may use the ports between 12500 and 12509 (including borders). Note that you of course can use the same port number for TCP and UDP services (so it is for example ok to use TCP port 12500 and UDP port 12500 at the same time).

Ant template

Ant is a Java based build tool that significantly eases the development process. If you have not installed ant yet, you should download and follow these instructions to do it.
We provide a template build file (build.xml) in which you only have to adjust some parameters and class names. Put your source into the subdirectory "src", move on the command line to the directory where the build file is located and simply type "ant" for compilation. Type "ant run-server" to start the server, "ant run-peerX" (with X being 1, 2 or 3) to start a peer.
Note that it's absolutely required that we are able to start your programs with these predefined commands!

Tricky parts

After hours of development you compile, start your engine and...
fail! No worries, here is some advice:

Binding problems

After starting your socket implementation, you receive a message comparable to Error 1:

Error 1

The address and port, you are trying to bind to is already in use. To solve these issues you can test your solution at home (and don't bind to ports used by other services ;), or you simply change the ports. Our policy to use your dslab account number prevents these errors! For example: dslab023 = Bind to port 10230

Further Reading Suggestions

• APIs:
  • IO: IO Package API
  • Concurrency: Thread API, Runnable API, ExecutorService API, Executors API
  • Java TCP Sockets: ServerSocket API, Socket API
  • Java Datagrams: DatagramSocket API, DatagramPacket API
• Tutorials:
  • JavaInsel Sockets Tutorial - Section 17.7, 17.8: German tutorial for using TCP sockets.
  • JavaInsel Datagrams Tutorial - Section 17.11: German tutorial for using datagram sockets.